feat(config): add codegraph.json "deprioritize" for ranking-only path down-weighting (#982) - #1463

The ranking half of #982, kept deliberately separate from the corpus-frequency discount in #1462 — the issue asks for the two to stay distinct because they have different semantics, and measuring confirmed they cover different shapes.

Why a second lever

matchesNonProductionDir hardcodes example/sample/fixture/benchmark/demo, so a peripheral tree only the project knows about — optional-skills/, scripts/, a generator's output dir — gets no de-prioritization at all. When helpers there carry generic symbol names (usage, status, run), an exact name match hands them a large bonus and they crowd out the product code that answers the query.

{
  "deprioritize": ["optional-skills/", "scripts/"]
}

Gitignore-style patterns, matched against project-root-relative paths — same contract as exclude, include, includeIgnored, and validated by the same warn-and-skip rules (a non-array value or a blank/non-string entry is dropped with a warning, never thrown).

This is a relevance lever, not a recall lever. exclude takes content out of the index; deprioritize leaves it fully indexed and findable and only stops it from winning. There's a test pinning exactly that — after de-prioritizing, getNodesByName('usage') still returns both helpers and a direct search for usage still surfaces them.

The part I got wrong first, and what the measurement showed

My first cut only added the patterns to the existing −15 path penalty, matching the built-in non-production dirs. It did not fix the issue's repro, and the numbers say why:

The path penalty is additive, and the signal it has to counter — the exact-name bonus — is additive and larger. No magnitude of a hand-tuned additive penalty fixes that shape robustly; it just moves the threshold.

So the lever now also targets the mechanism directly: a path the project de-prioritized is saying its symbol names are not the answer, so the exact-name bonus is damped there as well (0.75× — derived below; it started as a guessed 0.25×). Damped rather than zeroed, so the tree still ranks when it genuinely is what you asked for — the same "discount, don't erase" rule as the path penalty.

Worth flagging as a finding regardless of this PR: the built-in example//fixture/ de-prioritization has the same weakness. A usage() inside a hardcoded examples/ dir would out-rank product code today for exactly the same reason. This PR fixes it for user-declared paths; say the word and I'll extend the damping to the built-in set in a follow-up (I left it out to keep the diff to one behaviour).

Relationship to #1462

Complementary, not overlapping — this is the point of keeping them distinct:

Neither subsumes the other. They're on independent branches off main and touch scorePathRelevance's signature in the same region, so whichever lands second needs a trivial rebase — happy to do that in either order.

Two defects a review pass caught, now fixed

Both were real, both are pinned by new tests.

1. The setting needed a process restart to take effect. The matcher was built once in wireLayers(), which runs only from the constructor and reopenIfReplaced(). src/mcp/tools.ts keeps one CodeGraph per project root alive for the whole server lifetime — so a user editing codegraph.json, re-indexing, and seeing nothing change would reasonably conclude the feature was broken. exclude and include don't behave that way. The predicate now reads loadDeprioritizePatterns() per call (mtime-cached — one stat) and memoizes the compiled matcher on the pattern array's identity. There's a test that writes the config after opening the project; I confirmed it fails against the old code.

2. explore only half-honoured the setting — and explore is the surface this issue reports on. Both scorePathRelevance call sites in src/context/index.ts passed two arguments, so the lever never reached them; explore improved only incidentally, via the two sub-searches that route through searchNodes. Rows B, C and D of #982's own reproduction table are all codegraph explore, so shipping it search-only would have missed the reported case. Both sites now pass it.

What I deliberately did not join: explore's hard early-continue filters and its non-production budget cap. Those remove content from the result set, and deprioritize is a ranking lever by definition — exclude is the lever for putting something out of reach. Mixing the two would collapse the distinction the issue asks us to preserve.

The README claim was also overstated — it said this "extends" the built-in example//fixture/ list, when the built-ins get five mechanisms and this gets one. Narrowed to state exactly what it does.

Smaller items from the same pass: scorePathRelevance now takes a boolean rather than a predicate (the caller had already evaluated it, and it was being invoked twice per result); the predicate body is exception-guarded so a malformed path can never take a search down with it; the misplaced module const moved out from between two imports; and two test assertions that could pass vacuously on an empty result set were tightened.

Tests

__tests__/deprioritize-config.test.ts, 16 tests:

• parsing — default empty with no config; patterns kept verbatim and trimmed; non-array warns-and-skips; blank/non-string entries dropped while the rest survive; other config keys undisturbed
• ranking — a control asserting that without the config the helpers still take the top two ranks (the reported status quo), then that with the config product code outranks them
• recall preserved — the helpers stay indexed and directly findable
• no collateral — packages/core/util/strings.ts, which no pattern names, scores identically with and without the config
• config written after open — fails on the eager implementation, verified
• explore — the matcher is reachable from the context builder and classifies both trees correctly
• a query that genuinely targets the tree — bodyfat calc usage still ranks the de-prioritized helper first with product code competing, which is explore/query relevance: a generic token's exact name-match overboosts in peripheral dirs — follow-up to #746 #982's "discount, don't erase" edge case
• the damping constant is derived — 80 × SCALE − 15 must stay above the prefix arm's ceiling; one of the two tests fails at the originally proposed 0.25
• the two deliberate asymmetries — a path that is both test-like and de-prioritized is docked once, not twice; and the user penalty is not waived by a test-y query the way the inferred built-in one is (a standing project declaration outranks a heuristic — asserted so it reads as a decision, not an accident)

Regression suites, all green: context-ranking, explore-corroboration-ranking, explore-nl-stopword-collision, explore-result-count, explore-blast-radius, explore-output-budget, context, symbol-lookup, same-name-disambiguation, field-name-retrieval, search-query-parser, exclude-config, include-config, include-ignored-config, is-test-file — 165 passed. tsc --noEmit clean. README documents the key alongside exclude/include.

The A/B this repo gates on — now run, and what it did and didn't show

CLAUDE.md requires a scorer change to be A/B'd with scripts/agent-eval/*, ≥2 runs/arm. That gate was the honest blocker on this PR, so it is now run: ab-new-vs-baseline.sh (both arms codegraph-on — the only harness that isolates a retrieval change) plus run-all.sh for the with/without pass bar. sonnet / high, RUNS=2, baseline ref = this branch's merge-base (572d22b), target = a 62k-node django index carrying a codegraph.json the baseline build simply doesn't understand.

Headline: the agent-level A/B is a null result. The deterministic evidence is real and is what this section actually offers. Details below, including the two attempts that measured nothing.

Deterministic probes — the lever does what it claims, on a real corpus

django, 62,080 nodes, deprioritize: ["tests/"] (that tree is 68.4% of all nodes — a fair stand-in for a project whose peripheral tree dwarfs its product code).

40 queries, auto-generated from the 20 symbol names most concentrated in tests/ that also exist in first-party code — i.e. exactly the crowd-out condition, not queries chosen to flatter the change:

Instrument control: same build, same index, config present but with a pattern that matches nothing (no-such-tree/) → 0 of 40 queries changed, byte-identical rankings. So the deltas above are the lever, not run-to-run anything.

A concrete one — setup handling during request processing, top 10, 6 peripheral slots → 1:

codegraph_explore moves too, which matters because explore is the surface #982 reports on. With a core-dev-shaped config (["django/contrib/", "tests/"]), where is urlpatterns configured at startup returns 5 files, 3 of them django/contrib/** URL modules on the baseline; with the lever, 3 files, 1 peripheral — django/conf/__init__.py and django/core/management/commands/listurls.py take the freed budget, and the response drops 24.7k → 23.2k chars.

And the case that should not improve, stated plainly: how does the test runner set up the database before running tests — a query that genuinely targets the de-prioritized tree — still returns it (5 of 7 files, alongside django/test/runner.py). That is "discount, don't erase" working, not a miss.

0.25× → 0.75×: the number you asked me not to guess, now derived

You asked me to set these two numbers rather than guess. One of them turned out to be measurable, so I stopped guessing it and pushed the change.

Sweeping the damping constant on the same 40 queries, plus a recall check on 15 symbol names that exist only inside the de-prioritized tree (an on-target query for those has no first-party alternative, so the tree should still win):

Two things fall out:

1. 1.0 — the −15 path penalty alone — clears 14 of 88. That is the quantified version of the "additive penalty can't beat an additive, larger bonus" argument above, on a real corpus rather than the fixture.
2. 0.25 breaks the PR's own rule. The names that lose rank 1 there lose it to prefix matches: child → children@django/test/utils.py, parent → all_parents@django/db/models/options.py, method → method_decorator@django/utils/decorators.py. That is indefensible ranking behaviour regardless of this feature.

There's a bound that predicts it: nameMatchBonus's prefix arm tops out below 10 + 30 = 40, and a de-prioritized node also takes the −15 path penalty, so 80 × SCALE − 15 > 40 is what keeps a damped whole-query exact match ahead of a mere prefix match at any corpus shape. 0.75 clears it (45); 0.25 doesn't (5). And since 0.75 and 0.5 clear virtually the same crowd-out (39 vs 40 of 88), the deeper discount was buying almost nothing.

So the constant is now 0.75, derived, with two tests pinning the bound — one of them fails at the old 0.25, so it can't drift back silently. The other number, the −15 path penalty, I left alone: it's shared with the built-ins and changing it would move behaviour this PR doesn't own.

Agent A/B — null result, reported as such

ab-new-vs-baseline.sh, django + ["django/contrib/", "tests/"], question chosen after verifying with the probe above that it actually surfaces the de-prioritized tree (where are urlpatterns configured at startup…), RUNS=2 per arm, on the shipped 0.75 build:

Control repo (excalidraw, no codegraph.json → the lever is inert by construction, so any gap is pure variance): duration 38s [35–41] vs 41s [37–45], residual 31.5k [31.1–31.9k] vs 24.4k [17.8–30.9k]. That control puts the noise floor at roughly ±7k tokens of residual and ±5s of duration at n=2 — as large as every gap in the table above. An earlier pass of the same question on the interim 0.25 build even came out the other way (38s [33–43] vs 47s [45–49], residual 15.5k vs 21.6k). Two runs of the same question disagreeing on direction is the definition of a null result, and I'd rather say so than quote the favourable half.

Pass bar (run-all.sh, with vs without codegraph, same question, 2 invocations): with = 3–4 tool calls, 0–1 Read, 0 Grep, 2 explore calls, 39s/55s; without = 6–8 tool calls, 4–5 Read, 2–3 Bash, 40s/41s. The Read/Grep half of the bar is met comfortably; the "faster than without" half is a wash on this question (39 vs 40, then 55 vs 41). That measures codegraph as a whole, not this lever.

Two measurements I threw away, and why

Both are the same trap, and it's worth writing down: "the instrument didn't see it" is not "it isn't there."

1. First attempt: both arms made 0 codegraph calls. Given a flow question with Read/Bash available, the agent solved it by grep and Read in both arms — 8 tool calls, no explore. Any difference between those arms is about grep, not about ranking. Both arms then got an identical instruction to use codegraph_explore and not shell out; from then on every run is 2–3 explore calls and 0 Grep.
2. Second attempt: the question never surfaced the de-prioritized tree at all. "How does Django turn a URL into a view" returns handler and resolver files in both arms — no tests/, no contrib/. The lever had nothing to act on, so the arms differed only by noise (new looked worse: 88% vs 96% allocation efficiency). Discarded, and the question replaced with one the deterministic probe had already shown to be contaminated in the baseline.

Where that leaves the merge decision

The gate is procedurally satisfied — ab-new-vs-baseline.sh and run-all.sh, ≥2 runs/arm, isolated arms, disclosed methodology. But I'm not going to claim the agent-level win it was designed to detect: at n=2 on one repo the arms are inside the control's noise. What the measurement did buy is concrete: it caught that my own damping constant broke the "discount, don't erase" invariant, and replaced it with a derived one.

If a null agent A/B is disqualifying for a ranking lever that is off by default, say so and I'll close this. If the deterministic evidence is enough — the effect is user-opt-in, zero-diff when the key is absent, and provably no-op when the patterns match nothing — then the remaining decision is just whether you want the same damping extended to the built-in example//fixture/ set, which has the identical weakness.

Two small things I've deliberately left for the merge moment rather than doing now: a rebase onto current main, and the CHANGELOG entry (adding it to this branch's stale [Unreleased] section is a guaranteed conflict — I checked). Both are one command each once you tell me this is going in; I kept the head at the exact tree the numbers above were measured on.